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Abstract — In practical nonlinear filtering, the assessment of 
achievable filtering performance is important. In this paper, we 
focus on the problem of efficiently approximate the posterior 
Cramer-Rao lower bound (CRLB) in a recursive manner. By 
using Gaussian assumptions, two types of approximations for 
calculating the CRLB are proposed: An exact model using the 
state estimate as well as a Taylor-series-expanded model using 
both of the state estimate and its error covariance, are derived. 
Moreover, the difference between the two approximated CRLBs 
is also formulated analytically. By employing the particle filter 
(PF) and the unscented Kalman filter (UKF) to compute, simu- 
lation results reveal that the approximated CRLB using mean- 
covariance-based model outperforms that using the mean-based 
exact model. It is also shown that the theoretical difference 
between the estimated CRLBs can be improved through an 
improved filtering method. 

Index Terms — Posterior Cramer- Rao lower bound (CRLB), 
approximated CRLB, Fisher information matrix (FIM), non- 
linear dynamical system, Taylor series expansion. 

I. Introduction 

It is well known that optimal estimators for the nonlinear 
filtering of the discrete-time dynamic systems is an active 
area of research and that a large number of suboptimal 
approximated approaches were developed [1]. It is impor- 
tant to quantify the accuracy of estimates obtained for the 
design of algorithms such as the interacting multiple models 
(IMM) where weighted estimates from multiple estimators 
are simultaneously employed. 

During the past thirty years many attempts have been 
made to theoretically derive the achievable performance of 
nonlinear filters. Deriving performance bounds are important 
since such bound serve as indicators to measure system 
performance, and can be used to determine whether imposed 
performance requirements are realistic or not. 

For dynamical statistical models, a commonly used bound 
is the CRLB that has been investigated by various re- 
searchers: Van Trees [2] presented the batch form of a 
posterior CRLB for random parameter vectors and a pre- 
1989 review [3] summarized several lower bounds for nonlin- 
ear filtering, which heavily emphasized the continuous time 
case. Bobrovsky [4] applied CRLB to discrete time problems 
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and Galdos [6] generaUzed it to the multi-dimensional case. 
The main shortcoming of these formulations is the batch 
form of implementation resulting high computational loads. 
Tichavsky [7] was the first to derive a recursive CRLB for 
updating the posterior Fisher information matrix (FIM) from 
one time instance to the next while keeping the FIM constant 
in size. 

Subsequently, CRLB theory was extended to many ap- 
plications, e.g., introducing the CRLB to multiple target 
tracking [9], incorporating data association for tracking with 
the CRLB [10], target detection for the case having a 
detection probability less than unit [8], etc. 

It is well known that the matrices in recursive form of 
FIM, can only be theoretically determined by the true value 
of state. Unfortunately, we cannot obtain the true state online 
in practice, except in some well-designed experiments where 
true value of the state is given as a prior knowledge. There- 
fore we naturally focus on how to determine an approximate 
CRLB by using online state estimates (as opposed to the true 
state values). 

We have mainly two ways to approximate the CRLB [5]: 
1) Make full use of the first-two order moments of the state 
estimate, i.e., expectation and covariance, by incorporating 
them with the Taylor series expansion of the dynamics. 2) 
Combine the expectation of the state with the exact dynamic 
model directly. The first method use both estimates and is 
rather complex while the second method is considerably 
simple, but depends heavily on an exact model. The second 
method is mostly preferred in practice for its simpleness and 
is sufficient to obtain an usable approximated CRLB. 

The following question therefore needs to be addressed: 
By how much the CRLB employed the two kinds of approxi- 
mations diff'er from, and which one is a better approximation 
to the true CRLB. This is the main motivation of this 
investigation. In addition, determining the accuracy of the 
estimated CRLB by using a state estimate, rather than the 
true state under a recursive framework for a general nonlinear 
dynamics, has not been addressed previously. 

In this paper, we show how the state estimates can be ap- 
plied to determine the difference between the two estimated 
CRLBs. By using Monte Carlo simulations, we show that the 
proposed method achieve a satisfactory approximation, and 
the accuracy of estimated CRLB can be explicitly improved 
by increasing the accuracy of filtering. 



II. Problem Formulation 

A. Nonlinear Dynamical Model 

Consider the following discrete-time nonlinear dynamics 
with additive Gaussian noise: 



Zk = h;t(Xi:) + \k, 



(1) 

(2) 



where the nonlinear vector-valued functions f^ € R"^ and 
hk e R™^' be used to model the state kinematics and mea- 
surement respectively, and generally n> m. Xk e R"^' is the 
state vector, Zk e R'"^^ is the measurement vector, w^ G R"^' 
is a zero-mean white Gaussian process noise with known 
covariance Qk, and \it e R'"^' a zero-mean Gaussian white 
measurement noise with variance R^^. The initial state xq 
is assumed as a Gaussian distribution with mean xq and 
variance Pq. Moreover, a general accepted assumption like 
cov(xo,\k) = O,cov(xo,-Wk) - 0. 

B. Posterior CRLB 

Let Xk and Ck denote the unbiased state estimate and its error 
covariance at time instant k. We therefore have 



Ci = 4xix;]>J^:^ 



(3) 



where x^ = Xi:-Xk is the prediction error of state. J^' is 
the posterior CRLB (PCRLB), defined to be the inverse of 
FIM, Jk. The superscript (■)' in (O denotes the transpose 
of a vector or a matrix, and the inequality in ^ means 
that the difference Ca-J^' is a positive semidefinite matrix. 
From [7], [11] we know that the sequential FIM J^. can be 
recursively calculated by 



Df-D2i(Ji, + Dii)-iDi2 



Jo = £[-A:^°logp(xo)], 



(k > 0), 



D 



■ E[-Alllogp(xk+i\xk)], 



K = (of)' = E [-All^' logpixk^ilx,)] , 



D 



,22 



^ E[-Alll[logp(xk^i\xk)] + 
£[-A;^*;;iog;,(z^+i|x^+i)], 



(8) 



here let V and A be operators of the first and second-order 
partial derivatives, i.e., V^ = [i^ r ■■,■£;;]', Al = Vx(Vy)'. 
Note that all the above expectations are taken with respect to 
the joint probability density function (PDF) /7(xo:A:+i|zi:/t+i), 
where xo:A:+i and zi:A:+i denote all the states and measure- 
ments up to time k+l. 

III. Approximated Gaussian Form (AGF) of Nonlinear 
Dynamics 

According to CRLB theory, the derivatives in (|4]i should be 
evaluated at the true value of state x^^. Our final aim is to 
use the moments of state estimate instead of the true state to 
calculate the difference between the approximated PCRLBs, 
thus the FIM matrices (i.e., D|', D^^ and T)f- should be 
represented, therefore, the density function p{xk+i\xk) and 
p(zii+i\xii+i) from ([TJ and (|2| should be firstly formulated 
explicitly. 



A. AGF by the First-two Order Moment of State Estimate 

Assume that the first and second moment estimation of state 
Xk is known and given by Xk and P|[ = £ x^^x^Jzi ,t , and also 
assume that the distribution of x^^+i can be approximated by 
a Gaussian. We immediately have 



Xk+\ X N\xk+V, ^k+\, P^i]. 



(9) 



where xa:+i = £'[xa:+i|zi:A:] a: h + 'ik, in which fA: = iki^k), 
f/t = 5 2"=i e;f[S^ ,P[], e,- e R"^' denotes the /-th unit normal 
vector in column shape, and tr[-] denotes trace operation. 
S^ . = Vx^. Vxj./A-,,(xi:) is the Hessian matrix of /-th element 
fkA'^k) of the vector-valued function (ki'f^k)- Notation F^ = 
Vx;;f^(xi:) -\dfkj(xk)/dxj\ denotes the Jacobian matrix 
with nxn dimension, P^^j =P^+i +Q<r, P^+i = F' P[(F[)' H- 
5Z"=iZ;-=ie,e:fr[sf P[Sf Pf]. Similar to Q, the Gaussian 
form of the measurement Zk can be approximated by 



Zk~N[zk; Zk, Pi], 



(10) 



where the expectation z^ - E[zk\xk] ~ hk+hk, the covari- 
ance P^, a Rk + Pl, in which 6^ = 5 2"lie,fr[S^.P^], P^ = 
Fl'P^(F^' + ^i:;ii2:;L,e,e;,r[S^P^Shphj^^ xhe'term^s h,, 

F^ and S^. are similar to the definitions of f^, F^ and S[ . in 
^, respectively. 

B. AGF by the First Order Moment of State Estimate 

As an alternative to the approximation presented in Section 



be 


nil- A! we use the state estimate xt to represent xto-i 


and Zk. 




By denoting this version of representation as x^^j 


and z*. 


(4) 


we have 




(5) 


xi+i^M'^A+i' ^^' Q^_' 


(11) 


(6) 


zl a n[zI; h,, R; , 


(12) 


(7) 


where the definitions of f^: and h^: are same as that in 
IlII-Al 


Section 



IV. Approximated FIM 

A. The Case Using Mean and Covariance 

According to distribution of x<:+i and Zk in ^ and ( fTOl l. the 
log-PDF of state and measurement, given by x^^ and x^+i, 
can be respectively formulated by 

logp(xj:+i|xi:) = ci - -logdet[P^^i] - - [(xA:+i -Xk+i)' 

x(P^^i)'(x,+i-x,+i)], (13) 

logp(zA.+i|xA.+i) = C2 - -logdet[P^^j] - - [(Zk+i -Zk+i)' 

x(P^^i)'(z,+i-z,+i)], (14) 

where c\ and C2 are constants. Calculate the derivatives of 
\ogp{xk+\\xk) and \ogp(zk+i\xk+\) with respective to Xk and 
Xk+\ respectively, specifically we have 

Vx,_n [\ogp(xk+i\xk)] = -{Pl^^r\xk+i -Xk+i), (15) 



then consider the definitions of FIM in ©-([H]) and after 
algebra arrangement, finally we obtain 
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(PLi) 



(16) 



(17) 



\dzk- 



+1 



k+\ 



dx' 
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(PL,) 



dP^ 

dx' 



(PL,) 



k+l 



dP^ 



(18) 



It is explicit that the right hand of ( fTSI l and the second 
term on the right hand of (fTSl l is similar with that in [12]. 
We observe that all derivatives involved in (fTSt-lfTSll can be 
evaluated by using the mean and covariance of the state 
estimate instead of the true state. 

So far, based on the Gaussian model assumption, we 
formulate the matrices used by the PCRLB in (|4]i as above. 
In order to obtain the difference between the two kinds 
of approximated PCRLBs, matrices in (fT6ll-(fT8ll should be 
decomposed as shown in the follows. According to the well- 
known matrix inversion lemma [13], we have a simplified 
formulas as below 



(A + B)- 



■(AB-^A + A)-\ 



(19) 



where A, B are the nonsingular matrices, and the inversion 
of every matrix is assumed to exist. For the matrix D^', we 
can decompose the inversion of the covariance matrix P? 



k+l 



defined in (|9]l into two terms, (PL ,) ' = Q^,' -I'm where 



«P^ = [Qi:(Pp 'Qa + Qic\ . Substituting it and the expression 
of Xk+i into ( fTSI ), after some arrangements yield 



i=l j=l 



dt'k 



dfk 



^4 dxi, 



+2, 



(20) 



where 

.=1 7=1 
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dt'k 
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dx\ 
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For matrix D^^, we decompose (P^.^.)) ' = R^i - ^Lr 



where ^P^.^^ 



= [Ri+i(P^^j)-iR^+i+R^+i]" 
and the expression of i]^ into ( fTSl ) yields 



Substituting it 



x22 



D -q: 



;=i 7=1 



5h; 



1+1 



dx' 
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d\vk 



+1 
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dx' 
"\+l > 



+^11, (21) 






where 



^22 —~ 
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dx' 
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k+l 



dx' 
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d%. 
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dx 



k+l 



R-1 k+l m; 



dik 



+1 



^k+l 



dx' 
^^k+l 



k+l 



dx' 



For matrix D^^, substituting (P^+j) ' = Qj-' -'P* and x^^+i «; 
\k + \ into ( flTb yields 



i2_ ^f;^-i,f^f; , ^fr 



^ dx^' [dXk dXkj 



dt; 



'^^^Q" 



(22) 



2l2 



So after the above steps, we successfully rewrite the matrices 
D|^ Dp and D|^ into two parts respectively, then we 
submit expressions in (l20ll-(l22li into the definition of FIM 
in (HJi, using the matrix inversion lemma again, after some 
expansions and arrangements yield 



J k+l = 2^22 - ^'12 [Jk + ^11) 



s*2+n. 



(23) 



where 



n = S22-(Df)'(J^ + Dll)-lSi2- 

a) = [(ji+E;i)(E„)-'(j,+E;i) + (j,+s;,)l"\ 

B. The Case Using only Mean 

By comparing the mean-based Gaussian form presented in 
([TSIl-dTSll. we straightforwardly arrive at (use the superscript 
symbol * to distinguish with that in Section HV-Al l: D|_'* = 
Sjj, D|^* - S*2 and Dp* - 12^^^. Then substituting matrices 
of D*s into the definition of FIM in (|4| yields 

JLi = D'+i - (Dp*)'(J^ + D^'*)-'Df * ^ 0. (24) 

V. Difference Between the Two PCRLB Approximations 

Our final aim is to calculate the difference between the 
two approximated PCRLBs, where the one approximation 
employ the first-two order moment of state estimate and the 
other one only use the first order moment. 

Performing the matrix inversion lemma on the FIM J^^+i 
defined in ( |23] ) again, we get the PCRLB J^i directly 



'/t+i 







(n-'0-Hi)-'0 



(25) 



Explicitly the difference between the two kinds of approx- 
imated PCRLBs, defined by J 
formulated by 



1 

k+l 



'k+l 



•J^^p can be 



-It* 



j^^i = (n-'0+i)-'0-' = (n-'jLi + i)"'(JLi)"'' (26) 

where I denotes an identity matrix with appropriate dimen- 
sion. In Section [VT] Monte Carlo simulations show that the 
bound J^j) is always higher than the Jr'. , that is to say, the 
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Fig. 1. Comparison RMS en'ors of state estimation generated by two 
different estimators: Particle filter (PF) and Unscented Kalman filter (UKF). 
100 runs of Monte Carlo simulations and the initial number of particles is 
1000. 



Fig. 2. Comparison of the true posterior CRLB with the first type of 
approximations. The approximated PCRLB corresponds to the method of 
"exact model and expectation of state estimation", and were generated by 
Particle filter (PF) and Unscented Kalman filter (UKF). 100 runs of Monte 
Carlo simulation. 



J^^j is more closer to the true PCRLB than that of J^^j, 



of 



course this is for the case with finite number of particles. For 
the situation as sampling A^ tends to infinity, the convergence 
theoretically needs further investigation. 

VI. Experimental Results 

To evaluate the performance of the proposed algorithm, the 
following typical univariate nonlinear model [14] is studied: 



Xk =0.5ji:i:_i + - 



1 +x 



■ + 8cos[1.2(;t-l)]+w<:. 



k-l 



(27) 



20 



yi<^:^ + '^k, k^l,2. 



here using Wk ~ N{0, c^^^) denotes the process noise, and Vk ~ 
N{0,crl) is the measurement noise. Data was generated by 
using cr^^, = 1, cr^ = 5, and T - 50. The initial prior distribution 
was chosen as p{xQ) ~20xN{0,l)- 

For comparison purposes, we implemented two state es- 
timation methods: 1) The unscented Kalman filter (UKF) 
where it is not necessary to compute Jacobian matrices 
and the performance is accurate to the third-order term (in 
the Taylor series expansion) for Gaussian inputs, even for 
nonlinear systems. For non-Gaussian inputs, approximations 
are accurate to at least the second-order term [15]. 2) The 
particle filter (PF), where an initial sample size A^ - 1000 
is adopted, and 100 runs of Monte Carlo simulation are 
performed. 

Filtering accuracy by using the same trajectories is shown 
in Fig.l. Here the root mean square (RMS) error is used 
as an evaluation criterion. It should be firstly noted that for 
the PF the initial number of samples is generally chosen 
by trial-and-error and that its accuracy can be improved 
by increasing the sample size. Secondly, according to [14], 
the likelihood p(yk\xk) has a bimodal nature when Xk > 0, 
and this bimodality causes the state too acutely fluctuate 
and complicates to track using conventional filtering. The 
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Fig. 3. Comparison of the true posterior CRLB with the second type 
of approximations. The approximated PCRLB corresponds to the method 
of "Taylor expanded model and first-two moments of state estimation", the 
two estimators: Particle filter (PF) and Unscented Kalman filter (UKF), were 
employed. 100 runs of Monte Carlo simulation. 



RMS error in Fig [T] clearly reflects the effect of the nonlinear 
dynamic phenomena. 

Fig|2] shows the comparison of the true PCRLB and the 
approach of "exact model and mean of state estimation", 
which refers the recursive FIM formulated by (l24l) . We can 
see from the figure that there exists an explicit error between 
the true PCRLB and both approximations. The PCRLB 
corresponding to UKF is overall worse than the PCRLB 
generated by the PF. As expected, the true PCRLB is a lower 
bound (always lower than the approximations in all instants). 

In Figl3] the true posterior CRLB is compared with 
the approach of "Taylor series expanded model and first- 
two order moments of state estimation". This approach is 
performed by substituting Eqn.(fT6ll-(fr8ll into (HJi and using 




Fig. 4. Comparison of two kinds of theoretical gap of posterior CRLB, 
upper and lower plots corresponds to Unscented Kalman filter (UKF) 
and Particle filter (PF), respectively. As expected, the gap improve as the 
accuracy of filtering improves. 



first-two order moments of state estimation as parameters, 
we observe that both estimated PCRLBs are more accurate 
approximations compared with the true PCRLB. In many 
instants the PCRLB corresponding to PF is closer to the 
true PCRLB than the approximation using the UKF. Due 
to the acute nonlinearity of the system, the PCRLBs appear 
strongly oscillatory throughout the simulation. 

We can directly calculate the difference between the two 
PCRLB approximations: PCRLB in Fig|2] minus the corre- 
sponding one in Fig|3] However, as a theoretical analysis, 
we employ the formula in ( |26] | and the calculated results 
are presented in Fig|4] The PCRLB generated by the PF 
is generally more accurate throughout simulation. When the 
initial sampling used by PF was increased, the accuracy of 
its corresponding PCRLB was improved. 
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VII. Conclusion 

In this paper, we considered the problem of approximate 
calculation of CRLB by using Gaussian assumptions and 
the moments of state estimate instead of using true state. 
Two kinds of approaches were proposed: One was an exact 
model using the expectation of state estimate; the other was 
an approximated model using the expectation and covariance 
of state estimate. Furthermore, the difference between the 
two estimated CRLBs was formulated analytically. By using 
state estimators of PF and UKF, we compared the proposed 
approximations with true PCRLB. Simulation results demon- 
strated the significance and validity of our approach. 

References 



[1] B. D. O. Anderson and J. B. Moore, Optimal Filtering. Englewood 

Clifl's, NJ: Prentice-Hall, 1979. 
[2] H. L. van Trees, Detection, Estimation and Modulation Theory. New 

York: Wiley, 1968. 
[3] T. H. Kerr, Status of CR-like lower bounds for nonlinear filtering, 

IEEE Trans. Aerosp. Electron. Syst., Vol.25: 590-600, Sept.1989. 



